Aggregating Learned Probabilistic Beliefs

We consider the task of aggregating beliefs of sev­ eral experts. We assume that these beliefs are rep­ resented as probability distributions. We argue that the evaluation of any aggregation technique depends on the semantic context of this task. We propose a framework, in which we assume that nature generates samples from a 'true' distribution and different experts form their beliefs based on the subsets of the data they have a chance to observe. Naturally, the optimal ag­ gregate distribution would be the one learned from the combined sample sets. Such a formulation leads to a natural way to measure the accuracy of the aggregation mechanism. We show that the well-known aggregation operator LinOP is ideally suited for that task. We propose a LinOP-based learning algorithm, inspired by the techniques developed for Bayesian learning, which aggregates the experts' distributions represented as Bayesian networks. We show experimentally that this algorithm performs well in practice.